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1.  INTRODUCTION: 


We  will  establish  the  performance  signatures  of  Robotic  Surgery  Readiness  (RSR) 
through  tasks  on  the  da  Vinci  robotic  virtual  reality  simulator  by  testing  the  role  intervals 
of  inactivity  have  on  task  performance.  These  signatures  will  be  used  to  develop  a 
simulation  curriculum  that  brings  the  inactive  surgeon  to  RSR.  The  curriculum 
effectiveness  will  be  tested  in  the  operating  room  on  practicing  surgeons  performing 
patient  surgery  with  and  without  the  RSR  warm-up  curriculum.  We  will  enroll  surgical 
residents  and  faculty  for  hypothesis  testing.  Objective  technical  performance  and  Global 
Evaluative  Assessment  of  Robotic  Skills  (GEARS)  scoring  will  be  correlated  by  the 
Principal  Investigator  (Dr.  Thomas  Lendvay  -  UW)  and  Co-Investigator  (Dr.  Timothy 
Kowalewski  -  UMN).  Optimal  methods  for  extracting  surgeon  performance  metrics  from 
the  da  Vinci  Application  Programming  Interface  (API)  will  be  evaluated  and  developed 
through  collaboration  with  the  Intuitive  Surgical  Consultant  (Simon  DiMaio,  Senior 
Research  Manager).  We  will  deliver  practical,  automated  RSR  assessment  methods  and  a 
warm-up  curriculum  able  to  bring  a  robotic  surgeon  to  his/her  optimal  state  of  readiness 
before  patient  surgery. 
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2.  KEYWORDS 


Robotic  Surgery 
Readiness 
da  Vinci  Simulator 
Virtual  Reality 
Simulation  Curriculum 

GEARS  -  Global  Evaluative  Assessment  of  Robotic  Skills 
Surgical  Education 
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3.  ACCOMPLISHMENTS: 

What  were  the  major  goals  of  the  project? 

YEAR  1  (0-12  Months) 

1)  Study  design  and  skill  decay  model  construction,  supplies  purchasing  and  acquisition.  (0-3 
months)  Completion  date  9/30/2015,  major  supplies  purchased  thru  6/30/2016 

2)  Set-up  and  flow  within  and  between  simulation  centers.  (0-3  months) 

Completion  date  9/30/2015 

3)  Development  of  robust  methods  for  collecting,  merging  and  verifying 
simulator,  video  and  optical  tracking  data.  (0-6  months) 

Completion  date  12/31/2015.  Continued  checks  as  more  equipment  conies 
online.  Tool  motion  metric  capturing  technology  development  -  ongoing. 

4)  Subject  recruitment.  (3-9  months) 

Recruitment  has  begun  at  3  of  the  4  sites.  50%  complete 

5)  Skills  decay  testing.  (3-12  months)  Several  subjects  will  begin  in  the  next  quarter. 

6)  Independent  video  review  of  VR  simulator  criterion  performances  using  GEARS  tool.  (9-12 
months) 

7)  Analysis  of  performance  metrics.  (9-15  months) 

Deliverables:  Quantifiable  performance  signatures  of  robotic  surgery  skills  decay 
assessment.  Initial  analysis  of  data.  Preliminary  RSR  warm-up  curriculum. 

YEAR  2  (12-24  Months) 

1)  Finalize  and  validate  RSR  curriculum  and  benchmarks.  (12-15  months) 

2)  Intra-operative  RSR  warm-up  subject  recruitment.  (12-24  months) 

3)  RSR  curriculum  hypothesis  testing,  intra-operative  data  collection.  (15-24  months) 

4)  Independent  video  review  of  surgical  performances  using  GEARS.  (18-24  months) 

Deliverable:  Finalized  RSR  warm-up  curriculum,  initial  dataset  and  data  quality 
assessment. 

YEAR  3  (24-36  Months) 

1)  Continued  intra-operative  RSR  curriculum  hypothesis  testing.  (24-33  months) 

2)  Continued  independent  video  review  of  operative  performances  using  GEARS.  (24-33 
months) 

3)  Biostatistical  analysis  and  model  cross-validation.  (30-36  months) 

4)  Abstract  and  manuscript  drafting.  (33-36  months) 

Final  Deliverables:  Completed,  validated  RSR  warm-up  curriculum  and  assessment 
tools.  Methodology  for  quantifying  robotic  surgery  skills  decay.  Peer-reviewed 
publication,  presentation  at  national  meeting. 
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What  was  accomplished  under  these  goals? 

YEAR  1  (0-12  Months) 

1)  Study  design  and  skill  decay  model  construction,  supplies  purchasing  and  acquisition.  (0- 

3  months) 

Study  design  and  skills  decay  model  completed  by  month  3  while  acquiring  major 
equipment  has  taken  the  entire  first  year  but  is  now  complete. 

Computers  have  been  purchased  and  setup  for  all  sites  (UW,  MAMC,  VA,  FL  Hospital) 

2)  Set-up  and  flow  within  and  between  simulation  centers.  (0-3  months) 

As  subjects  are  recruited  they  will  complete  the  intake  demographics  questionnaire  and 
complete  the  proficiency  training.  Subject  identifiers/details  will  be  kept  at  each  site  and 
only  de-identified  data  will  be  collected  by  the  team  at  Minnesota  (UMN).  All  subjects 
will  be  given  a  unique  identifier  based  on  their  location.  Work  with  a  UW  biostatistician 
has  helped  move  the  data  intake  and  form  development  along.  RedCap  is  now  being  used 
to  collect  data  and  to  randomize  the  subjects.  We  are  getting  all  site  coordinators  trained 
on  RedCap.  (Figures  1  &  2,  Appendix)  All  de-identified  video  and  simulator  data  output 
is  being  collected  by  the  team  at  UMN  for  analysis. (Figure  3,  Appendix) 

3)  Development  of  robust  methods  for  collecting,  merging  and  verifying  simulator,  video 

and  optical  tracking  data.  (0-6  months) 

UMN  has  reworked  the  software  to  provide  high  definition  video  acquisition  and 
compression  (previous  versions  had  poorer  resolution  and  video  artifacts). 

UMN  has  successfully  installed  and  monitored  the  acquisition  of  data  from  all  sites.  The 
data  is  synchronizing  with  our  central  database  as  designed.  Collaborative  agreement 
between  UMN,  UW,  and  Intuitive  Surgical,  Inc.  is  on-going  to  extract  tool  motion 
metrics  from  da  Vinci  robot. 

4)  Subject  recruitment.  (3-9  months) 

Aim  1  Recruitment  (RSR  Curriculum  design)  has  begun  at  3  of  the  four  sites.  MAMC 
has  15  contacted,  9  subjects  consented  and  6  have  completed  their  proficiency  stage. 
UW/VA  has  45  contacted  with  none  proficient.  FI  Hospital  have  none  as  of  yet  (Florida 
Hospital  and  UW  needed  to  reconcile  contract  language  before  proceeding).  MAMC  has 
been  able  to  act  as  our  test  site  to  confirm  that  our  planning  has  worked.  This  has  allowed 
us  to  edit  forms  and  procedures  to  make  the  work  flow  optimally. 

5)  Skills  decay  testing.  (3-12  months) 

MAMC  has  6  ready  to  be  randomized  and  begin  Aim  1. 
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What  opportunities  for  training  and  professional  development  has  the  project 
provided? 

All  of  our  Aim  1  participants  are  provided  training  in  robotic  surgery  simulation  activities 
in  order  to  meet  proficiency.  This  has  been  accomplished  through  peer  and  one-on-one 
training  with  an  expert.  The  Proficiency  Training  introduces  novices  and  hones 
experienced  clinicians  in  robotic  object  transfer,  suturing,  management  of  the  third  working 
arm,  camera  and  instrument  clutching  skills.  We  have  not  provided  “Professional 
development”  opportunities. 

How  were  the  results  disseminated  to  communities  of  interest? 

Nothing  to  report. 


What  do  you  plan  to  do  during  the  next  reporting  period  to  accomplish  the  goals? 

YEAR  1  (0-12  Months) 

3)  Development  of  robust  methods  for  collecting,  merging  and  verifying 

simulator,  video  and  optical  tracking  data.  (0-6  months) 

UMN  continues  to  renegotiate  the  tri-party  collaborative  agreement  with  Intuitive  Surgical 
and  University  of  Washington.  This  was  unavoidable  but  currently  does  not  impede  the 
progress  of  proficiency  testing  phase  or  Aim  1. 

4)  Subject  recruitment.  (3-9  months) 

Subject  recruitment  has  begun.  Invitation  emails  have  been  sent.  Proficiency  training  has 
begun  at  MAMC,  UW  and  VA.  Get  Florida  Hospital  up  and  mnning. 

5)  Skills  decay  testing.  (3-12  months) 

Skills  decay  testing  will  begin  with  the  consented  subjects.  Data  acquisition  will  be 
continually  monitored  by  UMN  team. 
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4.  IMPACT: 


What  was  the  impact  on  the  development  of  the  principal  discipline(s)  of  the  project? 

We  have  developed  a  video  and  data  capture  system  that  allows  remote  software  updates 
on  each  site’s  computer.  (Figure  3,  Appendix)  This  has  minimized  the  need  for  any  on-site 
software/hardware  servicing.  Furthermore,  a  workable  user-interface  was  developed  so 
that  each  site’s  coordinators  can  seamlessly  capture  video  and  upload  data. 

What  was  the  impact  on  other  disciplines? 

A  method  for  reliable  seamless  video  capture,  data  tagging,  and  storage  has  a  universal 
application  in  any  training  and  skills  assessment  programs. 

What  was  the  impact  on  technology  transfer? 

Nothing  to  report. 

What  was  the  impact  on  society  beyond  science  and  technology? 

Nothing  to  report. 
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5.  CHANGES/PROBLEMS:  ; 


Changes  in  approach  and  reasons  for  change 

Nothing  to  Report 

Actual  or  anticipated  problems  or  delays  and  actions  or  plans  to  resolve  them 

As  this  is  a  multisite  project  problems  were  anticipated.  We  had  some  delays  in  IRB 
approvals  at  the  VA  but  this  was  resolved  and  the  IRB  was  approved.  We  had  some  delays 
in  funding  some  of  the  sites.  Due  to  the  nature  of  funding  a  military  site  and  the  need  for  all 
funds  to  be  used  within  that  fiscal  year  we  had  to  bring  back  the  money  allocated  to  MAMC 
and  fund  any  manpower  hours  required  centrally.  The  funding  contract  with  Florida 
Hospital  took  longer  than  anticipated  but  was  able  to  be  worked  out.  Specifically,  the 
University  of  Washington  (a  state-funded  institution)  was  prohibited  from  providing  a  sub¬ 
contract  to  an  institution  that  did  not  have  an  Equal  Opportunity  Employment  (EOE)  clause 
(Florida  Hospital).  This  was  resolved  and  the  sub-contract  was  granted. 


Changes  that  had  a  significant  impact  on  expenditures 

Nothing  to  Report 

Significant  changes  in  use  or  care  of  human  subjects,  vertebrate  animals,  biohazards, 
and/or  select  agents 

Nothing  to  Report 

Significant  changes  in  use  or  care  of  human  subjects 

Nothing  to  Report 

Significant  changes  in  use  or  care  of  vertebrate  animals. 

Nothing  to  Report 

Significant  changes  in  use  of  biohazards  and/or  select  agents 

Nothing  to  Report 
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6.  PRODUCTS: 

Publications,  conference  papers,  and  presentations 

Nothing  to  Report 

Journal  publications. 

Nothing  to  Report 

Books  or  other  non-periodical,  one-time  publications. 

Nothing  to  Report 

Other  publications,  conference  papers,  and  presentations. 

Nothing  to  Report 

Website(s)  or  other  Internet  site(s) 

Nothing  to  Report 

Technologies  or  techniques 

Nothing  to  Report 

Inventions,  patent  applications,  and/or  licenses 

Nothing  to  Report 

Other  Products 

Nothing  to  Report 
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7.  PARTICIPANTS  &  OTHER  COLLABORATING  ORGANIZATIONS 


What  individuals  have  worked  on  the  project? 

Name:  Thomas  Lendvay 

No  change 

Name:  Karen  Edwards 
No  change 

N ame :  Anna  French 
No  change 

Name:  Prof.  Tim  Kowalewski 
No  Change 

Name:  Sara  Teller 

Project  Role:  Project  Manager/Site  Coordinator 

Researcher  Identifier  (e.g.  ORCID  ID): 

Nearest  person  month  worked:  1 

Contribution  to  Project:  Project  management  and  subject  management 

Has  there  been  a  change  in  the  active  other  support  of  the  PD/PI(s)  or  senior/key 
personnel  since  the  last  reporting  period? 

Nothing  to  Report 


What  other  organizations  were  involved  as  partners? 

Nothing  to  Report 
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8.  SPECIAL  REPORTING  REQUIREMENTS 
COLLABORATIVE  AWARDS: 

Nothing  to  Report 


QUAD  CHARTS: 

Submitted 
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APPENDICES 


9. 


Record  ID 


1)  Age: 

•  must  provide  value 


•  must  provide  value 


2)  Gender: 

Male  C  Female  O  other 

*  must  provide  value 

reset 

3)  Please  select  the  category  that  most  accurately  reflects 

Junior  (R1-R3) 

the  participant's  seniority: 

Senior  (R4-R8) 

•  muet  provide  value 

Attending  (Fellow/Staff) 

reset 

4)  If  Staff,  how  many  years  have  you  been  out  of  training? 

Currently  a  Fellow 

reset 

S)  If  not  Staff,  please  select  your  current  year  in  training: 

6)  Surgical  speciality: 

Urology  Gynecology  General  Surgery 

*  must  provide  value 

reset 

7)  Experience  with  robotic  surgery  (#  cases  as  primary 

surgeon,  all  time): 

•  must  provide  value 

If  no  experience,  please  record  0 

8)  Experience  with  laparoscopic  surgery  (#  cases  as 

primary  surgeon,  all  time): 

*  most  provide  value 

9)  Handedness 

Left  Right  Ambidextrous 

•  must  provide  value 

reset 

10)  Do  you  have  musical  experience? 

Yes  “  No 

*  must  provice  value 

reset 

Brass 

C  Percussion 

11)  If  yes,  what  type  of  Instrument  do  you  play? 

Strings 

Woodwinds 

Keyboard 

12)  If  yes,  how  many  years  have  you  played  an  Instrument? 

r  Daily 
Weekly 

13)  If  yes,  how  often  do  you  play  an  instrument?  Monthly 


Rarely 

I  no  longer  play  any  instruments 

reset 


14)  Have  you  ever  been  deployed  or  on  civilian  leave  for  an 
extended  amount  of  time? 

*  must  provide  value 

Yes  "  No 

reset 

15)  Date  of  return  from  last  deployment  civilian 
leave/sabbatlcal  (if  applicable): 

®  Today  1  '<  ‘M* 

16)  Duration  of  deployment/leave/sabbatical  in  months: 

17)  Did  you  perform  surgery  while  you  were  deployed  or  on 
leave? 

Yes  No 

reset 

18)  Percent  of  time  in  the  operating  room  (not  In  an 
administrative  role)  while  deployed  or  on  leave: 

Report  value  between  0  and  100 

19)  Days  since  last  performed  robotic  surgery 

•  must  provide  value 

If  no  experience,  please  record  365. 

20)  Days  since  last  simulation/practice 


*  must  provide  value 

Figure  1:  Demographic  form  in  RedCap 
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Current  instrument:  Proficiency  Training 


Return  to  edit  view 


NOTE:  Please  be  aware  that  branching  logic  and  calculated  fields  will  not  function  on  this  page.  They  only  work  on  the 
survey  pages  and  data  entry  forms. 


This  form  should  be  completed  for  each  module  where  proficiency  goal  is  reached. 
Modules  1-3:  Each  task  must  be  completed  with  a  goal  of  overall  score  of  90%  or  more. 


Ring  and  Rail  2 
Score  reached? 

*  must  provide  value 

Score: 

Date  90%  score  reached: 

Match  Board  3 
Score  reached? 

*  must  provide  value 

Score: 

Date  90%  score  reached: 

Suture  Sponge  3 
Score  reached? 

*  must  provide  value 

Score: 

Date  90%  score  reached: 


Yes  C  No 


13  I  Today  ]  Y-M-0 


Yes  C  No 


13  Today  1  Y-M-0 


Yes  C  No 


13  I  Today  ]  Y-M-0 


Module  4:  The  Cont.  Suture  task  must  be  completed  with  proficiency  goal  of  78  seconds  or  less. 


Continuous  Suture 


Time  achieved? 

*  m  jst  provide  value 

Time  (in  seconds): 


How  was  this  module  timed? 


Yes 


No 


Simulator 

Researcher  stopwatch 
Peer  stopwatch 


Date  that  time  goal  was  reached: 
Proficiency  achieved? 

*  must  provide  value 

Date: 

•  rrust  provide  value 


13  I  Today  ]  Y-M-0 


Yes 


No 


13  Today  )  Y-M-0 


Figure  2:  Proficiency  Form  in  RedCap 
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Figure  3:  Video  Capture  Interface 
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